feat: track process lineage for telemetry#216
Draft
mabdinur wants to merge 7 commits into
Draft
Conversation
Contributor
Author
|
Blocked by the release: DataDog/libdatadog#1822 |
…-io] The new [patch.crates-io] section in instrumentation/Cargo.toml was not reflected in the lock file, causing `cargo build --locked` to fail in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Point to the published crate versions from DataDog/libdatadog#1989: libdd-data-pipeline 3.0.1 → 3.1.0 libdd-telemetry 4.0.0 → 5.0.0 libdd-capabilities-impl 1.0.0 → 1.1.0 These versions include the APIs required for instrumentation session ID propagation (TelemetryInstrumentationSessions, NativeCapabilities, session_id fields on telemetry Config). With them published, the [patch.crates-io] git overrides are no longer needed and are removed from both Cargo.toml and instrumentation/Cargo.toml. Note: this commit will not build until libdatadog#1989 merges and the new crate versions are published to crates.io. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
Author
Next steps before mergingThis PR currently carries The patch overrides have now been dropped in the latest commit, pointing instead at the versions that will exist once DataDog/libdatadog#1989 merges and the crates are published:
This PR will not build until the following sequence completes:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Propagates instrumentation session IDs across process boundaries so that telemetry and trace exports from child processes (spawned or forked) can be linked back to the root process in the Datadog backend.
New module
telemetry_session(public API):sessions_from_runtime_id(runtime_id)— builds aTelemetryInstrumentationSessionsstruct by reading_DD_ROOT_RS_SESSION_ID/_DD_PARENT_RS_SESSION_IDfrom the process environmentinstall_process_lineage_env(runtime_id)— writes_DD_ROOT_RS_SESSION_IDinto the current process env on first call (idempotent, must be called before threads are spawned)lineage_env_for_spawn(runtime_id)— returns the two env pairs to pass to a child processextend_command_env_with_lineage(cmd, runtime_id)— convenience wrapper forstd::process::CommandTelemetryInstrumentationSessions,ENV_ROOT_RS_SESSION_ID,ENV_PARENT_RS_SESSION_ID— re-exported from the crate rootWiring:
Config::buildcallsinstall_process_lineage_envso the root session ID is stamped into the env before any threads startmake_telemetry_workerpassessession_id/root_session_id/parent_session_idtolibdd_telemetry::config::ConfigDatadogExportercallsset_telemetry_instrumentation_sessionson the trace exporter builderAdapts to libdatadog v33.0.0:
TraceExporter<C>is now generic; addedpub type TraceExporter = LibddTraceExporter<NativeCapabilities>to keep the rest of the codebase unchangedwait_agent_info_readyis nowasync; wraps it in atokio::current_threadruntime in thetest-utilspathlibdd-capabilities-implworkspace dependencyDependency pin (
[patch.crates-io]): all six libdatadog crates are overridden withtag = v33.0.0because crates.iolibdd-data-pipeline 3.0.1predates thetelemetryfeature andTelemetryInstrumentationSessions. The same patch is mirrored ininstrumentation/Cargo.toml.Motivation
Enables the Datadog backend to correlate telemetry and traces emitted by a parent process with those from its child processes, using the
dd-session-id/dd-root-session-id/dd-parent-session-idheaders already supported by libdatadog.Additional Notes
ddtrace-rs-clientworkspace inDataDog/system-testsresolves libdatadog from crates.io and has no[patch.crates-io]. A companion PR to system-tests must add the matching patch section toutils/build/docker/rust/parametric/Cargo.tomlbefore the parametric suite can build with this branch.libdd-data-pipelineincludes thetelemetryfeature (libdatadog v33+).